Technical Field
A “Concurrent Projector-Camera” uses an image projection device in combination with one or more digital cameras and optional lighting to enable various techniques that provide visually flicker-free projection concurrently with real-time image or video capture in the same space in a manner that eliminates video feedback from the projection into the real-time image or video capture.
Background Art
If two or more people are trying to share a common desktop where each has a camera and a projector, a video feedback loop will be created causing the video to become useless in a manner similar to audio feedback which results in high pitch squealing. This problem has been referred to as “visual echo.”
More specifically, “visual echo” represents the appearance of the projected contents viewed by a camera back into the projection over time. This problem is analogous to audio or acoustic echo in telephone communications where the person speaking hears a delayed version of his own voice that may become increasingly corrupted by ongoing echo over time. As with acoustic echo cancellation, visual echo cancellation has been performed using a variety of techniques, some of which require significant computational resources.
For example, one conventional visual echo cancellation scheme for sharing a virtual workspace or virtual whiteboard between separate locations uses a setup in which the captured video at each location contains local writings or user gestures (i.e., the “foreground”) along with projected contents representing the shared workspace. Therefore, if the captured video is simply broadcast to each separate location, there will be a feedback loop that will distort the projected image. In fact, after only a few frames, some parts of the projected image will become saturated while some parts of the real (i.e., local) writing will appear to have a ghosting effect. This problem is addressed by using an off-line calibration procedure that records a geometric and photometric transfer between the projector and the camera in a look-up table. Then, during run-time, projected contents in the captured video are identified using the calibration information and computationally suppressed, therefore achieving the goal of canceling visual echo.
Another conventional approach to visual echo cancellation synchronizes a fast switching DLP projector and a camera, so that the camera takes an image only when the projector is off or showing a specific pattern. This effectively interleaves the operation of the projector and camera in a time-sequential manner using default projector timings. The use of a fast-switching DLP projector avoids visual flicking. While this approach avoids visual echoes by interleaving projection and image capture operations, it requires careful analysis of the DLP projector color wheel and mirror flip timing for synchronizing camera activation timing in a manner that ensures that images are only captured when the DLP projector is either off or is projecting a known pattern. It has been suggested that such systems are difficult to implement and to accurately synchronize. A related approach simply prevented one or more entire frames from being projected to provide a time during which images could be captured by a co-located camera.